Array enhanced stochastic resonance and spatially correlated noises 
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We discuss the role of spatial correlations of the noise in the array enhanced stochastic resonance. 
We show numerically that the noises with negative correlations between different sites lead to signif- 
icantly larger values of the signal-to-noise ratio than the uncorrelated noises or noises with positive 
correlations. If the noise is global, the system displays only the conventional stochastic resonance, 
without any array enhancement. 
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I. INTRODUCTION 

Stochastic resonance (SR) y| is the best-known exam- 
ple of the constructive role of noise. The SR is a phe- 
nomenon in which the response of a system is optimized 
by the presence of a specific level of noise. It has been 
detected in so many seemingly different systems that it 
has been claimed to be "an inherent property of rate- 
modulated series of events" 2] . It has been recently sug- 
gested that the functioning of important natural devices, 
e.g., communication and information processing in neu- 
ral systems or subthreshold signal detection in biologi- 
cal receptors, rely on phase synchronization rather than 
stochastic resonance [3J, but this does not exclude the 
possibility that some natural devices may rely on the SR 
or that effective artificial detectors that use this feature 
may be constructed and operated. 

It has been observed that the SR gets enhanced if an 
array of similar nonlinear elements collectively responds 
to the same signal. This phenomenon has been termed 
the array enhanced stochastic resonance (AESR). It was 
first observed in chains of nonlinear oscillators 3 and 
later, mostly without explicitly using the term AESR, 
in arrays of FritzHugh-Nagumo model neurons in 
ion channels [6(, in ensembles of nondynamical elements 
with internal noise independent on the incoming signal or 
modulated by it fjj, in multilevel threshold systems @, 
in a globally coupled network of Hodgkin-Huxley model 
neurons and, recently, in arrays of Josephson junc- 
tions [ljj. The experimental work on detecting the SR 
in the mammalian brain tissue |ll| should also be men- 
tioned as a whole array of neurons was stimulated. The 
accumulated knowledge of systems exhibiting the AESR 
is now considerable, but there are still important issues 
that need to be clarified. We want to address two such 
points in the present paper. 

First, in nearly all papers mentioned above, the spe- 
cific dynamics — the "internal" dynamics of systems such 
as nonlinear oscillators or model neurons, or dynamical 
coupling between various elements — played an impor- 
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tant role. It is thus not entirely clear what features of 
the AESR are generic, and what depends on the details 
of interactions. We will discuss the AESR in arrays of 
nondynamical threshold elements. Such elements, first 
introduced in Ref. and later discussed in Refs. [l3| . 
are known to display the SR when subjected to a periodic 
subthreshold signal with an additive noise. The AESR, if 
indeed present in such an array, is not obscured by any ef- 
fects resulting from a specific dynamical model, and only 
its generic features should show up. The interesting re- 
search presented in Ref. [?J already considered the AESR 
in ensembles of nondynamical elements, but we want to 
re-examine this problem because of the special treatment 
of noises in Ref. — see a discussion below. 

Second, the role of spatial correlations between the 
noises has been almost neglected in the existing research 
on the AESR. The noises discussed were local and un- 
correlated in Refs. [J, la, 111 la 

El, 

global in Ref. .11], and 
in the form of a mixture of uncorrelated local and global 
noises m Refs. @. As correlations between various 
noises can result in very interesting physical phenomena, 
ranging from drastic changes in activation rates Il4lll5| . 
through current reversal in Brownian ratchets |l6j . to 
an effective nullifying one of the noises ^tJj an d many 
others, it is interesting to examine their impact on the 
AESR. The authors of Ref. [j| claimed that spatial cor- 
relations between the noises weakened the response of an 
array of neurons, but the spatial correlations discussed 
there were induced by the presence of a global noise. We 
will show that spatial correlations between local noises 
can have a constructive effect. 

This paper is organized as follows: In Section II we 
present numerical results for arrays of uncoupled thresh- 
old elements. In Section III we discuss the role of spatial 
correlations between the noises in a chain of nonlinear 
oscillators. Concluding remarks are given in Section IV. 



II. UNCOUPLED THRESHOLD ELEMENTS 

A single threshold device fires whenever the signal on 
its input exceeds the threshold. Here we consider an 
array of N such devices, acting in parallel in response to 
a common signal; the average (or integrated) output of 
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all individual devices is taken as the output of the whole 
array: 



ll if Asin(ut + <f>) + r)i(t) > 1 
5iW = otherwise (la) 



1 W 



(ib) 



Here u> is the frequency of the signal (we take u> = 2ir), 
4> is the initial phase, and A is the amplitude; we take 
A = 0.8 to make the signal subthreshold. g(t) is the 
output of the array and rji{t) are the noises. We take 
them to be zero- mean Gaussian white noises (GWNs), 
possibly spatially correlated: 



<>*(*)> = o, (0) 



5(t-t>). (2) 



The matrix C = [Cy] represents spatial correlations of 
the noises. It must be symmetric and positively definite. 
a is a parameter controlling the intensity of the noises. 
This system is similar to, but different from that dis- 
cussed in the first part of Ref . , where the local noise 
was added to the output of each element after the ele- 
ment had decided whether to fire in a response to the 
signal contaminated by a global noise. Thus, the output 
of a single element was not binary. Rather than that, it 
could assume, in principle, any value, positive or nega- 
tive. Moreover, as all elements received identical inputs, 
they all fired or not fired in unison. After an appropri- 
ate scaling, the collective output of the whole array was 
equivalent to a binary series contaminated by a GWN. 
The local noises in Ref. were, by assumption, spa- 
tially uncorrelated. In our approach, the internal noise is 
added to the signal at each site before the threshold el- 
ements make their decisions whether to fire or not. The 
internal noise represents fluctuations in the connections 
or environmentally induced random perturbations, but 
the output of every threshold element remains binary. 

We also start with uncorrelated, or local, noises, Cy = 
Sij. We calculate the signal and the noises with a time 
step h — 1/32 and calculate the output of the nondynam- 
ical system Q. We use Marsaglia algorithm to gen- 
erate the GWNs; we use the famous Mersenne Twister 
|19| as the underlying uniform generator. For each ar- 
ray, we collect a time series of 4096 elements, calculate 
its power spectrum and calculate the signal-to-noise ratio 
(SNR): 




FIG. 1: The AESR for the nondynamical system JTJ. The 
curves correspond, bottom to top, to the arrays of lengths 
1, 2, 4, 8, 16, 32, 64, 128, 256, 512, and 1024, respectively. 
The noises acting on different elements of the arrays are not 
correlated, CV, — Sij. 



noise strengths as the array size doubles. We can see 
that the AESR is not a result of any specific dynamics 
but is present also in arrays of uncoupled elements. This, 
together with previous results for coupled systems, shows 
that the AESR is a generic feature of arrays of elements 
that individually display the SR. 

Note that we have observed the AESR for a local noise. 
If the noise were global, Vi,j : rjiit) = rjj{t) (or Cy = 1), 
no array enhancement would be possible. Indeed, for a 
global noise, each element of the array receives identi- 
cal input, and the collective output of the whole array is 
identical with that of a single element. The array does 
not display the AESR, but only the conventional SR. 
This simple argument shows that it is the differences be- 
tween the noises at different sites that cause the AESR. 
This observation leads to the question of the role of spa- 
tial correlation between the noises. Qualitatively speak- 
ing, noises with large correlations are "nearly identical" 
and the enhancement of the SR should be small. The en- 
hancement of the SR should increase as the correlations 
decrease. We will now see that this is indeed the case. 

Let C = GG T be the Cholesky decomposition [2(j 
of the correlation matrix C and let rj(t) be a vector of 
spatially uncorrelated, zero mean GWNs, (rji(t)Vj(t')) — 
a 2 5ij 6(t — t'). Then r\ = Grj has correlations of the form 
@. In the following, we will consider only correlations 
between the nearest neighbors in the array of threshold 
elements. Specifically, 



SNR = 10 log 



power density at the signal frequency 



10 



background power density 



(3) 

For each N and c\ we average the results over 512 re- 
alizations of the noises and initial phases of the signal. 
Final results are presented in Fig. ^ an d their interpre- 
tation is clear: The SNR increases significantly for all 



Cij — 



1 if i = j , 
ci if \i-j\ = l, 
otherwise , 



(4) 



where \c\ \ ^ 0.5 in order to keep the matrix C positively 
definite. For each specific value of c±, the Cholesky de- 
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FIG. 2: The effect of spatial correlations on the AESR. The 
curves correspond, top to bottom, to ci = —1/2, —1/4, 0, 
1/4, and 1/2, respectively. The array consists of 128 threshold 
devices. 



composition needs to be performed only once; later, dur- 
ing the simulation, vectors of independent Gaussian vari- 
ables are generated and multiplied by the Cholesky fac- 
tor G. Since for a tridiagonal correlation matrix C, the 
Cholesky factor has only two non-zero elements per row, 
generating GWNs with nearest-neighbor correlations is 
only twice as computationally expensive as generating 
independent GWNs, or even less than that given the fact 
that generating the independent Gaussian variables is the 
most computationally intensive part of the procedure. 

We now simulate the system Q in the manner de- 
scribed above for various values of c\. Results for an 
array of N = 128 elements are plotted in Fig. [5] We 
can see that noises with positive spatial correlations lead 
to a smaller enhancement of SR than the independent 
noises. On the other hand, the SNR increases as c\ be- 
comes negative and approaches —0.5. The effect of nega- 
tive correlations (anticorrelations) becomes larger as the 
noise strength increases. Similar results have been ob- 
served for arrays of different lengths. 

Clean signals are rare in nature. Suppose that the in- 
coming signal is contaminated by a noise that cannot be 
controlled. To improve detection of this signal, we pass it 
through an array of threshold elements and we apply ad- 
ditional noise that we can control to each of the elements. 
The external noise acts here as a global noise; the addi- 
tional (local) noise is not correlated to the global noise. 
The above discussion suggests that the additional noise 
should have negative spatial correlations. If the global 
noise is weak (below the peak of the ordinary SR), the 
additional anticorrelated noise can significantly enhance 
the SNR, Fig. [3J However, if the global noise is large, 
the enhancement provided by the array is only marginal. 
This is similar to the result reported in Ref. [!j , where the 
presence of a strong global noise markedly deteriorated 
the performance of the system studied. 

It is important to understand the "microscopic" mech- 



FIG. 3: The AESR as a function of the strength of the global 
(external) noise oo and the strength a of the additional noise 
applied to the array. The additional noise has spatial corre- 
lations in the form with ci = —0.5, and the array size 
N = 8. 

anism responsible for the AESR. In the conventional SR, 
the threshold element may occasionally misfire or miss 
some peaks of the signal, cf. Fig. 0^,. Also the shape of 
the incoming signal is not resolved by the outgoing binary 
signal. When an array of such elements acts in parallel, if 
one of the elements makes a mistake (fires when the sig- 
nal is low or fails to fire when the signal is strong), other 
elements that are not positively correlated with it are not 
likely to repeat the mistake (Figs. 0b and c). This effect 
is even stronger when the other elements are negatively 
correlated with the one that makes the mistake: nega- 
tive correlations between noises at different sites tend to 
correct the mistakes, while positive correlations tend to 
repeat them. As a result, the shape of the incoming signal 
is resolved much better. This does not lead to a visible 
increase in the height of the signal peak in the power 
spectrum, but it does lead to a significant decrease of the 
noise background, Fig. ^ bottom row. Thus, lowering of 
the flat noise background is primarily responsible for the 
increase in the SNR. 

On a more formal level, let p be the probability that 
a single element fires. This probability depends on the 
current phase of the incoming signal, on the signal's am- 
plitude, and on the noise level. If exactly k out of N 
elements fire, the array's output equals g = k/N. If the 
noises at different sites are mutually independent (uncor- 
related), the probability of such an event is given by the 
binomial distribution: 

P N (g(t) = k/N) = (J)p k (l -P)^ • (5a) 

In an array twice as large with other parameters the 
same, exactly 2k elements should fire to produce the same 
output. The probability of this event is 

P 2N (g(t) = k/N) = [ 2 2 N k )p 2k ^~P) 2(N - k) ■ (5b) 




sr 10 



FIG. 4: Upper row: typical realizations of signals outgoing from a single threshold device (panel a), from an array of 8 devices 
with independent noises (panel b), and from a similar array with anticorrelated noises (panel c). Positive values of the 
incoming periodic signal are shown by a broken line. Bottom row: Power spectra averaged over 512 realizations of the noise. 
Noise parameters are the same as in the corresponding panels in the upper row. In all cases a — 0.375. 



The distributions JSJ have the same expectation values 
k/N, but for all values of < p < 1 the distribution (|5b|l 
is narrower than the distribution (|5a|) . More importantly, 
the distribution (|5b|) . corresponding to the larger array, 
allows for a more dense output, with values (k ± \)/N 
being more probable than (k±l)/N etc. Consequently, in 
larger arrays, wildly "wrong" outputs are less probable. 
This leads to lowering of the noise background and to an 
increase of the SNR. 

If the noises are spatially correlated, the probability 
that exactly k elements fire is no longer given by the bi- 
nomial distribution. We have not been able to derive an 
exact formula for this probability, but the general mech- 
anism of the SNR increase with the array size appears to 
be similar to that for the uncorrelated noises. Note that 
for the case presented in Fig. 0] introducing the maxi- 
mal negative correlations between the nearest neighbors 
lowers the background by a factor of the order of 2, in- 
creasing the SNR by 101og 10 2 ~ 3 dB, or about 10% of 
the total. 



III. A COUPLED SYSTEM 

In order to verify whether similar effects are present 
in coupled systems, we consider the same system that 
was discussed in Ref. where the AESR was first ob- 
served. Namely, we consider a chain of overdamped, cou- 
pled, nonlinear (double-well) oscillators 



x n kx n k x' n -\- s( K x n —i 2x n x n -\-\^j 

+A(smcjt + 4>) + r) n (t), (6) 

where k = 2.1078, k' = 1.4706, A = 1.3039 (these are 
the values used in Ref. Q), u) = 2tt : <f> is the initial phase 
of the signal, and rj n (t) are GWNs, possibly spatially 
correlated according to 

If the system responds to the external periodic stim- 
ulation, the central oscillator switches between the two 
wells in synchrony with the stimulation. We show results 
for the "extreme" cases of a global noise, a local (uncor- 
related) noise, and noises maximally correlated and an- 
ticorrelated between the nearest neighbors. The noises 
are generated by the algorithm presented in the preced- 
ing Section. We choose a chain of a modest length of 
N = 33, integrate the equations JBJ numerically using 
the Heun scheme [2lJ with a time step h — 1/64 and 
analyze the behavior of the central oscillator. We filter 
the analog time series to generate the time series of ±1, 
reflecting which well the oscillator is in. From the power 
spectrum of the binary time series we calculate the SNR 
and average over 64 realizations of the noises and the ini- 
tial phases of the signal. Results are presented in Fig. 

As we can see, for low noises the SNR curves display 
rather wild oscillations, but for larger noises the effect is 
much the same as for the nondynamical system discussed 
above: As the correlations decrease, the SNR maximum 
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FIG. 5: Stochastic resonance for the coupled system © with a global noise, a maximally correlated noise (ci = 1/2), an 
uncorrelated (local) noise (ci = 0), and a maximally anticorrelated noise (ci = —1/2). Panel (a) £ = 1, panel (b) e = 8, panel 
(c) e = 16. 



shifts towards higher noise levels. The maximal value, as 
well as values for large noise levels, are the largest for the 
noise maximally anticorrelated between the nearest oscil- 
lators and the smallest for the global noise. This effect 
grows as the coupling strength increases. A more thor- 
ough analysis shows that for large noise intensities, and 
increase in the SNR is again achieved mainly by lowering 
the noise background. Note that in case of the global 
noise, the response practically does not change with the 
coupling strength. In this case, all the oscillators receive 
identical inputs and there is no AESR, but only the or- 
dinary SR, exactly as in the nondynamical system. 

To understand the mechanism that is responsible for 
the AESR in this case, observe that the oscillators ex- 
change energy via the elastic coupling. We calculate the 
change in the elastic energy between two neighboring os- 
cillators that occurs during a time interval h > 0, AE — 
E n (t + h)- E n (t), where E n (t) = \e (a? n _i(t) - x n (t) f . 
To the lowest order in h, x n (t + h) x n (t) + fup n (t), 
where (p has contributions from the elastic interactions, 
the nonlinear part of the potential, the external signal, 
and the noise. If either the coupling constant e is large, or 
the oscillator happens to be in the vicinity of the barrier 
between the wells, the nonlinear part may be neglected. 
Straightforward calculations show that the noises con- 
tribute to the expectation value (AE) a term equal to 
ea 2 (l — Ci). We can see that noises with negative cor- 
relations maximize this contribution: Noises with nega- 
tive correlations tend to pull the neighboring oscillators 
in the opposite directions, thus maximizing the energy 
transfer between the oscillators and providing one of the 
oscillators with the extra energy needed to cross the bar- 
rier. This simple argument explains why the AESR grows 
when the coupling strength increases, why the AESR is 
larger for anticorrelated noises, and why the system with 
a global noise, corresponding to (r] n Vn±i) — \Vn)> does 
not display the AESR. Note that if the coupling were 
repulsive, the situation would be the opposite. 

The effects that the correlations have on the spatiotem- 
poral synchronization of the system © will be discussed 
separately. 



IV. DISCUSSION 

We have discussed the AESR in arrays of nondynam- 
ical threshold elements. We have not observed any sat- 
uration of the SNR curves that was reported previously 
in Ref. |5j- However, the input signal in that reference 
was aperiodic and the output SNR is not a natural mea- 
sure for such signals | 22| . This problem has been already 
discussed |23|. We have shown in the present paper that 
for arrays of nondynamical elements, noises with nega- 
tive spatial correlations lead to an enhancement of the 
AESR. In case of positive spatial correlations the AESR 
is weaker than for the independent noises, and arrays 
with a global noise do not display the AESR, but only 
the ordinary SR. This happens because detectors with 
negative correlations tend to correct each other's mis- 
takes, while positively correlated detectors tend to re- 
peat the mistakes. As a result, negatively correlated de- 
tectors better resolve the shape of the incoming signal. 
The mechanism of enhancing the SNR relies on lowering 
the noise background, not on elevating the signal peak. 
Note that we have analyzed these facts for spatial cor- 
relations between the nearest neighbors only. We expect 
that noises with long-ranging negative correlations would 
resolve the shape of the incoming signal even better; this 
point will be discussed elsewhere. It should be noted, 
though, that long-ranging correlations are more costly to 
generate. 

These results are, superficially, in disagreement with 
those of Ref. , where it has been claimed that spatial 
correlations between the noises diminished the positive 
effect of passing the signal through an array of model neu- 
rons. This discrepancy is easily solved: In Ref. |9|, each 
neuron was subjected to a superposition of a subthresh- 
old periodic signal, a local GWN noise, and a global 
Orstein-Uhlenbeck noise. The local noises were mutually 
uncorrelated and the spatial correlations resulted solely 
from the presence of the global noise. It was the strong 
global noise that was responsible for the deterioration 
of the output signal. We have observed a similar effect 
— see Fig. [3] above and the subsequent discussion. The 
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beneficial effects of negative spatial correlations reported 
here result from the correlations between the local noises. 

Our results suggest that from a technological point of 
view, not only the additive noises should be incorporated 
into the design of multi-component signal-detection sys- 
tems, as was already suggested in Ref. but also that 
these noises should have, if possible, negative spatial cor- 
relations to further improve the system's ability to detect 
weak signals, even with a weak global noise present. 

We have also shown that spatial correlations of the 
noise act similarly in the AESR in a coupled system. 
While we have discussed this for the specific system © 
only, we have shown that the AESR is enhanced by noises 
with negative spatial correlations due to the nature of the 
attracting harmonic interactions between the individual 
oscillators, regardless of the properties of the nonlinear 
part, provided the nonlinear part admits the conventional 
SR. For such interactions, positive spatial correlations of 
the noise reduce the AESR and a global noise eliminates 
it altogether, leaving only the ordinary SR, just like in 
the case of the AESR in arrays of nondynamical elements. 
As harmonic interactions between different particles are 
ubiquitous in many physical models, we expect similar 
phenomena to happen in a variety of situations. This 



analysis has also some interesting consequences for the 
interpretation of experimental results with interacting 
agents (particles, oscillators, detectors etc.) and a global 
noise, like those reported in Ref. [ll|: Even though one 
cannot examine a single agent and has to excite a group 
of interacting ones, with only a global noise added to the 
signal, the SNR response of the whole array is the same 
as that of a single agent. 

Previous research on systems that display the AESR 
and our present results let us conclude that the follow- 
ing features appear not to depend on the details of the 
dynamics: (i) For periodic subthreshold inputs, the SNR 
is systematically enhanced as the size of the array grows; 
(ii) Negative spatial correlations between the local noises 
provide further enhancement of the SNR; (iii) The SNR 
enhancement is mainly due to lowering the noise back- 
ground, not due to increasing the signal peaks; (iv) Posi- 
tive spatial correlations reduce the enhancement; in par- 
ticular, a purely global noise eliminates the array en- 
hancement altogether. The detailed shape of the SNR 
curves, their slope, locations of the maxima, depend on 
particulars of the system studied and on properties of the 
input signals. 
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